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ABSTRACT 

Objectives: Assessing the quality of cancer care (QoCC) 
has become increasingly important to providers, 
regulators and purchasers of care worldwide. The aim of 
this study was to develop evidence-based quality 
indicators (Qls) for colorectal cancer (CRC) to be applied 
in a population-based setting. 
Design: A comprehensive evidence-based literature 
search was performed to identify the initial list of Qls, 
which were then selected and developed using a two- 
step-modified Delphi process involving two 
multidisciplinary expert panels with expertise in CRC care, 
quality of care and epidemiology. 
Setting: The Qls of the clinical cancer care (QC 3 ) 
population-based project, which involves all the public 
and private hospitals and clinics present on the territory of 
Canton Ticino (South Switzerland). 
Participants: Ticino Cancer Registry, The Colorectal 
Cancer Working Group (CRC-WG) and the external 
academic Advisory Board (AB). 
Main outcome measures: Set of Qls which 
encompass the whole diagnostic-treatment process of 
CRC. 

Results: Of the 149 Qls that emerged from 181 sources 
of literature, 104 were selected during the in-person 
meeting of CRC-WG. During the Delphi process, CRC-WG 
shortened the list to 89 Ql. AB finally validated 27 Qls 
according to the phase of care: diagnosis (N=6), 
pathology (N=3), treatment (N=16) and outcome (N=2). 
Conclusions: Using the validated Delphi methodology, 
including a literature review of the evidence and 
integration of expert opinions from local clinicians and 
international experts, we were able to develop a list of Qls 
to assess QoCC for CRC. This will hopefully guarantee 
feasibility of data retrieval, as well as acceptance and 
translation of Qls into the daily clinical practice to improve 
QoCC. Moreover, evidence-based selected Qls allow one 
to assess immediate changes and improvements in the 
diagnostic-therapeutic process that could be translated 
into a short-term benefit for patients with a possible gain 
both in overall and disease-free survival. 
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INTRODUCTION 

Research on quality of cancer care (QoCC) 
performed during the last decade has 
demonstrated that the increase in knowledge 
on treatments with proven efficacy does not 



Article focus 

■ Quality of Cancer Care (QoCC) studies on specific 
quality indicators (Qls) developed worldwide since 
the late 1990s showed both a continuous improve- 
ment of oncological care provided by the clinical 
structures involved and an increased availability of 
specialised care in the considered areas. 

■ This study aims to define evidence-based Qls for 
colorectal cancer (CRC) care, in order to favour a 
feasible evaluation of the oncological diagnostic- 
therapeutic process from a population-based 
cancer registration and data collection point of 
view. 

Key messages 

■ Qls should be defined, developed and tested 
with scientific evidence-based rigour in a careful 
and transparent manner, taking into account 
their degree of relevancy, validity, reliability and 
feasibility. 

■ The selected CRC Qls can be applied in a 
population-based setting, implying the inclusion 
of the elderly, considering age as an extremely 
important determinant of treatment. 



directly translate into optimal delivery of 
such treatments to patients. Moreover, accu- 
mulating evidence suggests that underuse 
and overuse of care may occur for patients 
with cancer. 1 2 In addition to survival ana- 
lysis, to evaluate and compare quality of care 
at the population-based level, the assessment 
of QoCC has become increasingly important 
to providers, regulators and purchasers of 
care, owing to the growing demand for ser- 
vices, rising costs, constrained resources and 
evidence of variation in clinical practice. 3 

QoCC studies and structured programmes 
on specific quality indicators (Qls) have 
been developed worldwide since the late 
1990s, showing both a continuous improve- 
ment of oncological care provided by the 
clinical structures involved and an increased 
availability of specialised care in the consid- 
ered areas. Most of these studies have been 
implemented at the regional level on a 



Bianchi V, Spitale A, Ortelli L, etal. BMJ Open 201 3;3:e00281 8. doi:1 0.1 1 36/bmjopen-201 3-00281 8 



1 



Quality indicators of clinical care in colorectal cancer 



ARTICLE SUMMARY 



Strengths and limitations of this study 

■ To develop CRC Qls, we used a formal iterative process, the 
RAND/UCLA Appropriateness Methodology widely diffused and 
validated within other QoCC research. The selected Qls are rep- 
resentative of the main steps of the diagnostic-therapeutic 
process. 

■ Owing to the evidence that research studies demonstrated that 
single-discipline panels select different indicators than do multi- 
disciplinary panels and to maximise the applicability of CRC Qls, 
we constituted two panels of experts, a local CRC Working Group 
and an external national/international academic Advisory Board 
(AB), who could offer a multidisciplinary perspective on practice 
and also guarantee that the selected Qls and their results would 
be comparable with national and international data. 

■ Possible limitations of the current work are the following: 

- The level of evidence found in the literature. This situation is 
common to many aspects of healthcare, and it was the reason 
that the expert panel methodology was developed— specifically, 
to identify the processes that are most likely to be valid mea- 
sures of quality when the highest level of evidence is not 
available. 

- The literature selection could have missed some relevant arti- 
cles. However, members of the CRC Working Group were 
likely to be very familiar with the literature, and had the 
opportunity to suggest other indicators based on their experi- 
ence and literature search; in this way, we believe to have 
integrated the best research evidence with clinical expertise. 

- The feasibility of measuring indicators in terms of data col- 
lection and calculation. However, both the CRC Working 
Group and AB were concerned about the feasibility, validity 
and reliability of clinical data collection, necessary for the 
calculation of each single indicator at the population-based 
level. In fact, in order to warrant an accurate measurement, 
those indicators reaching more than 70% of the agreement, 
confirming their scientific and clinical value, but evaluated at 
least by one of the experts as not feasible and difficult to be 
collected at the population-based level, were definitely 
excluded. In this way, we have overcome the feasibility limit. 



diagnosis-treatment process in the colon-rectum, pros- 
tate, uterus, ovary and lung cancers in the territory of 
Canton Ticino (South Switzerland) . 

Colorectal cancer (CRC) is an important health issue 
worldwide. It is the most common malignancy in Europe 
(excluding non-melanoma skin cancers) and the second 
most common in terms of cancer-related mortality. 9 In 
Switzerland, CRC is the second and third most frequent 
tumour in women and men, respectively. About 4000 
CRC cases are diagnosed annually, corresponding to a 
European age-standardised incidence rate equal to 49.4 
and 30.6 cases/ 100 000 inhabitants in men and women, 
respectively, and representing 11% of all tumours. 10-12 
CRC is the third leading cancer cause of death in 
Switzerland, with approximately 1600 deaths/year, corre- 
sponding to a European age-standardised mortality rate 
equal to 18.5 and 10.6 cases/100 000 inhabitants in men 
and women, respectively. With a 5-year survival probabil- 
ity equal to 60%, Switzerland is the country with the 
most favourable prognosis in Europe. 13 A recent Swiss 
report with follow-up to 2009 showed an additional 
5-year survival increase to 62%. 11 

The aims of the QC 3 project were as follows: (1) to 
define and confirm evidence-based QoCC indicators for 
the tumour localisations cited above, in order to favour a 
feasible evaluation of the oncological diagnostic-thera- 
peutic process from a population-based cancer registration 
and data collection point of view; (2) to define and imple- 
ment at the regional level standards of care for each QoCC 
measure, in terms of minimum and target requirements. 
In the present report, we will describe the initial part of the 
QC 3 project, meaning the process followed to identify the 
panel of specific QoCC indicators for CRC, as well as the 
list of QoCC indicators identified and approved both by a 
dedicated Working Group of local healthcare providers 
and by an external independent Advisory Board (AB) , in a 
perspective of data collection feasibility by a population- 
based cancer registry. 



territory with uniform legislative, health and geograph- 
ical characteristics, increasing the likelihood of recruit- 
ment of involved clinicians. 1 4-7 

Until now, in Switzerland, no population-based study 
on QoCC with a prospective design has been implemen- 
ted. In addition to the yearly renewed international 
guidelines for each type of cancer, there is still a need to 
evaluate the real conditions of care in the community. 
Population-based Cancer Registry data are therefore 
essential to describe and reflect the real world and 
routine care as well as to provide regular feedback to 
healthcare workers and decisionmakers about the man- 
agement of a disease in daily practice and those treat- 
ments that are routinely prescribed and/or effective in 
all patient groups. 8 Moreover, Cancer Registries repre- 
sent an independent observatory, thus assuring a fair 
evaluation service and avoiding any conflicts of interest. 

We therefore implemented the Qls of the clinical 
cancer care (QC 3 ) project, focusing on QoCC about the 



MATERIAL AND METHODS 

The QC 3 project is a prospective, descriptive study on 
QoCC implemented in a population-based setting; it is 
performed by the Ticino Cancer Registry over a 3-year 
time period (2011-2013) on the territory of Canton 
Ticino (South Switzerland). In this paper, we focus on 
the initial part of the project: the identification of the 
CRC Qls which will be used to evaluate QoCC about 
CRC in our region. 

Qls for CRC were developed involving a local expert 
panel, named the QC 3 Colorectal Cancer Working 
Group (CRC-WG). Elected members, selected on the 
basis of their expertise and on their daily clinical involve- 
ment in CRC care, were contacted to have their interest 
confirmed in being involved. The final QC 3 CRC-WG 
included two pathologists, four gastroenterologists, two 
oncologists, three surgeons, two radiologists, two radi- 
ation oncologists and one nuclear medicine specialist, 
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making up a total of 16 panellists, all working in the 
public hospitals or in the private hospitals and clinics of 
Canton Ticino. 

Published studies and references were identified 
through a comprehensive search on PubMed/MEDLINE, 
using initially specific strings/expressions, such as the fol- 
lowing: "quality of care OR quality indicators AND colorec- 
tal cancer", "diagnosis OR diagnostic AND quality 
indicators AND colorectal cancer", "pathology OR patho- 
logical AND quality indicators AND colorectal cancer", 
"surgery OR surgical AND quality indicators AND colorec- 
tal cancer", "radiation oncology OR radiotherapy AND 
quality indicators AND colorectal cancer", "chemotherapy 
AND quality indicators AND colorectal cancer", "surveil- 
lance OR follow-up OR outcome AND quality indicators 
AND colorectal cancer", "preoperative care OR periopera- 
tive care OR intraoperative care OR postoperative care 
AND colorectal cancer", "population-based AND quality 
indicators AND colorectal cancer". For each of the identi- 
fied candidate QIs, we performed a systematic literature 
review to identify the highest level of evidence supporting 
the validity of that QI for articles published from 1990 
onwards. The reference list of included articles were also 
examined to identify any additional article that had not 
been identified in the MEDLINE search. We included all 
the peer-reviewed articles, excluding case reports, letters, 
abstracts or editorials. If evidence at the highest level were 
limited or absent, then lower levels of evidence were evalu- 
ated. For example, if data were not available from rando- 
mised controlled trials, cohort or case-control studies, case 
series and expert opinion or clinical guidelines were 
reviewed. A selection of already approved QIs provided by 
the American Society of Clinical Oncology (ASCO), the 
National Comprehensive Cancer Network (NCCN), the 
National Initiative on Cancer Care Quality (NICCQ), the 
Quality Oncology Practice Initiative (QOPI) and the 
Florida Initiative for Quality Cancer Care (FIQCC) were 
included in the evaluation list, with the aim to transfer 
them from the clinical setting to the population-based 
setting. 1 2 4 7 1 *" 20 

The initial QIs list emerged from 181 sources of litera- 
ture, and it was proposed to the CRC-WG in the context 
of an in-person meeting held at the very beginning of 
the process. The list was then left to the QC 3 CRC-WG's 
evaluation for a period of 2 weeks. The participants were 
asked to provide a whole opinion with written comments 
about those QIs considered pertinent for the assessment 
of CRC care quality, to suggest additional QIs not 
already included in the list and to delete those QIs con- 
sidered to be not suitable. In order to make the selec- 
tion and evaluation easier, the QIs were subdivided into 
chapters recalling the Donabedian's 21 and the National 
Initiative for Cancer Care Quality schemes: diagnosis 
and staging, pathology, treatment, follow-up, outcome. 2 

Delphi round 1 

The QIs selection was performed by using a 
two-step-modified Delphi process. 22 The initial list of 



QIs, reanalysed by the QC 3 CRC-WG, was formatted as a 
questionnaire, where for each indicator was specified 
the numerator, the denominator and the sources of evi- 
dence from which it was extracted. The questionnaire 
was distributed by regular mail to the QC 3 CRC-WG, so 
to maintain it anonymously, along with a stamped, 
addressed return envelope and an attached letter with 
the deadline date of 2 weeks from the receipt and 
instructions for voting. Respondents were asked to rate 
each QI adopting the RAND Appropriateness 
Methodology (scales 1-9, l=extremely inappropriate; 
9=extremely appropriate), according to the selection cri- 
teria of relevance, scientific soundness (validity, reliabil- 
ity, comparability) and feasibility (precise definition and 
specification, data feasibility, reliability of data collec- 
tion). 23 - 25 Each QI was judged as validated if it reached 
a strong consensus for acceptance (>70% of the QC 3 
CRC-WG rated the QI with a vote >7), discarded if it 
reached a strong consensus for exclusion (>70% of the 
QC 3 CRC-WG rated the QI with a vote <3) and as a 
standby if there was an unclear consensus (4< votes <6), 
which implies an eventual in-person meeting. 

Delphi round 2 

The Delphi Round 2 questionnaire was performed with 
the same modalities as in the first round and enclosed 
the frequency distribution of the round 1 votes, allowing 
the panellists to eventually alter their responses, in the 
light of colleagues' assessments. 23 

AB review 

The list of selected QIs derived from the two Delphi 
rounds was then submitted to an independent external 
national/international academic multidisciplinary AB, in 
order to get an additional evaluation on the suitability 
of QIs as 'quality' indexes according to the criteria 
shown in the previous paragraph. The intent was to 
achieve at least one health professional for each spe- 
cialty. The AB included one pathologist, one gastroenter- 
ologist, two oncologists, two surgeons, one radiologist, 
one radiation oncologist, one nuclear medicine special- 
ist and one epidemiologist, making up a total of 10 
experts in CRC care (see Acknowledgements); all the 
panellists were involved daily in CRC care and they had 
been contacted with the same modalities of the QC 3 
CRC-WG. The selected QIs as well as the corresponding 
literature sources were distributed to the AB as an elec- 
tronic form where their opinion about QIs were 
expressed both as megatrends (ie, response yes/no to 
the suitability of each QI) and as eventual additional 
comments. 26 We considered every single QI as finally 
approved by the AB if it achieved >70% of the agree- 
ment (ie, >70% of respondents should have answered 
'yes'). Besides the vote ('yes' vs 'no'), the panellists had 
the chance to comment on a single QI from a 
population-based cancer registration and data collection 
point of view. Therefore, those QIs reaching more than 
70% of the agreement, confirming their scientific and 
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clinical value, but evaluated by at least one of the 
experts 'not completely feasible and difficult to be col- 
lected at the population-based level', were definitely 
excluded. 

RESULTS 

The QI selection process began in January 2011 and 
ended in December 2011. 

Participation of CRC-WG members throughout 
the process was high: 15 (100%) participated in the 
in-person meeting, 12 (80%) completed both the 
Delphi rounds 1 and 2. The Delphi round 1 question- 
naire respondent time was in the range of 18-60 days, 
while for the round 2, the delay time was in the range of 
8-55 days; these delays and the time for recruitment of 
the AB influence the long time spent on this part of the 
project. 

Figure 1 summarises the entire process used to select QI 
for CRC care. The literature search produced 181 citations 
dealing with CRC QoCC, also including the already vali- 
dated QIs provided by ASCO, NCCN, NICCQ QOPI and 
FIQCC. 1 2 4 7 14-20 From this search, we initially selected a 

Literature search 

149 QIs 

Seek and nomination of the multidisciplinary 
QC 3 CRC-WG 



In-person meeting of the QC 3 CRC-WG 
104 QIs 

Delphi process: questionnaires (round 1 and 2) 

89 QIs 

Seek and nomination of the 
national/international multidisciplinary AB 



Validation of quality indicators 
by the AB 



27 definitive QIs: 

diagnosis (N=6) 
pathology (N=3) 
treatment (N=1 6) 
outcome (N=2) 

Figure 1 Process used to select quality indicators for 
colorectal cancer care. AB, Advisory Board; QC 3 CRC-WG, 
QC 3 Colorectal Cancer Working Group; QI, Quality Indicators. 



total of 149 QIs, which were proposed to the CRC-WG in 
the context of the initial in-person meeting. The following 
discussion and revision reduced the list to 104 QIs before 
the modified Delphi process started; these QIs were 
divided into the following areas: diagnosis and staging, 
pathology, treatment, follow-up and outcome. After the 
whole Delphi process, the list was shortened to 89 QIs, dis- 
tributed as follows: diagnosis and staging (N=16), path- 
ology (N=20), treatment (N=38), follow-up (N=10) and 
outcome (N=5) . The QIs finally underwent the AB's evalu- 
ation; this last step, according to the procedure described 
in the methods, shortened the final list to 27 QIs: diagnosis 
(N=6), pathology (N=3); treatment (N=16), follow-up 
(N=0) and outcome (N=2). Table 1 reports detailed infor- 
mation for each QI: (1) QI description; (2) criteria for 
patients' inclusion in the numerator and denominator; (3) 
list of the necessary medical documentation that should be 
collected by the Cancer Registry to extract the needed and 
relevant information to build the specific QI, such as the 
report of the endoscopy, the pathology report of the biopsy 
and/or surgical resection, the preoperative radiological 
reports (eg, CTand MRI), the surgery report, the tumour 
board documentation, the oncological report, the radio- 
therapy report and database/documentation of the 
regional Office of Population Registry Rosters for the 
assessment of patients' vital status (for outcome QIs); 
(4) QI rationale and (5) related references. 

DISCUSSION 

In the preliminary phase of the QC 3 project shown in 
this paper, we developed a panel of evidence-based CRC 
QIs which are suitable for implementation in a 
population-based setting. 

To develop the QC 3 QIs, we used a formal iterative 
process, the RAND/UCLA Appropriateness Methodology 
widely diffused and validated within other QoCC 
research. 23 24 Owing to the evidence that research studies 
demonstrated that single-discipline panels select different 
indicators than do multidisciplinary panels and to maxi- 
mise the applicability of QC 3 CRC QIs, we constituted a 
working group which could offer a multidisciplinary per- 
spective on practice, including specialists, professionals, 
clinicians and researchers coming from both public and 
private hospitals. 27-33 Moreover, we have used a further val- 
idation step enrolling an independent national/inter- 
national academic AB. This choice was due to the aim of 
measuring QoCC within a Swiss region, with a point of view 
on the population-based data collection and evaluation, 
and of obtaining results which will be comparable with 
national and international data. We believe that the expert- 
ise and multidisciplinary representativeness of the QC3 
CRC-WG and of the AB will surely increase the quality, 
acceptance and translation of QIs into daily clinical 
practice. 

The selected QIs are representative of the main steps of 
the diagnostic-therapeutic process. The diagnosis QIs 
reflect the importance of a preoperative evaluation 
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Table 1 Quality indicators of colorectal cancer care according to diagnostic-therapeutic process (diagnosis, pathology, treatment — surgery, chemotherapy and 
radiotherapy — and outcome) and tumour site 



Clinical 
domain 



Site Quality indicator 



Numerator 



Denominator 



Medical documentation Rationale 



Reference 



Diagnosis C&R Proportion of patients 
(n=6) with colorectal cancer 

and diagnosis based on 
symptoms vs screening 
vs accidental finding 



Number of patients with 
colorectal cancer whose 
diagnosis is based on 
symptoms, defined as 
appearance or 
persistence of clinical 
events and signs, such as 
rectal bleeding, occult 
blood in stool, weight loss 
with no apparent cause, 
general abdominal 
discomfort, bowel 
obstruction, change in 
bowel habits, constant 
tiredness, anaemia 



Number of patients 
with colorectal 
cancer 



Request form of 
endoscopic examination, 
endoscopy and surgical 
pathology reports, 
reports/discharge letters 
coming from all hospital 
units/departments (ie, 
surgery, medicine, 
radiation oncology, 
medical oncology) 



Assessment of the patient's 
take charge 



18 34-38 



Number of patients with 
colorectal cancer whose 
diagnosis is based on 
screening, defined as 
regular examination, such 
as faecal occult blood test 
(FOBT) or colonoscopy in 
asymptomatic patients 

Number of patients with 
colorectal cancer whose 
diagnosis is an accidental 
finding following 
examinations or therapies 
for other diseases (eg, 
hospital admission for 
other causes...) 



Continued 



Table 1 Continued 



Clinical 
domain 



Site Quality indicator 



Numerator 



Denominator 



Medical documentation Rationale 



Reference 



C&R Proportion of patients 
with colorectal cancer, 
evaluated by preoperative 
colonoscopy 



Number of patients with 
colorectal cancer who 
have been evaluated by a 
preoperative colonoscopy 



Number of patients 
with colorectal 
cancer undergoing 
surgery 



Endoscopy report, 
request form of pathology 
examination, pathology 
report of endoscopy 



Planning of further 
diagnostic procedures and 
treatments, 

comprehensiveness of 
diagnostic and staging 
evaluation 



7 16 18 40-41 



Proportion of patients 
with rectal cancer and 
description of the tumour 
localisation (distance ab 
and) in the endoscopic/ 
pathological 
documentation 



Number of patients with 
rectal cancer who have the 
description of the tumour 
localisation, in terms of 
distance ab ano, in the 
endoscopic/pathological 
documentation 



Number of patients 
with rectal cancer 
undergoing 
endoscopy 



Endoscopy report, 
request form of pathology 
examination, pathology 
report of endoscopy 



Planning of further 
diagnostic procedures and 
treatments, 

comprehensiveness of 
diagnostic and staging 
evaluation 



C&R Proportion of patients 
with colorectal cancer 
and requests for an initial 
CT and/or an MRI 
examination completed 
by clinical information 
according to the ACR 
guidelines 

Proportion of patients 
with low rectal* cancer 
undergoing pelvic MRI of 
staging 



Number of patients with 
colorectal cancer for 
which the request of an 
initial CT and/or an MRI 
examination is completed 
by clinical information 
according to the ACR 
guidelines 

Number of patients with 
low rectal* cancer who 
have undergone a pelvic 
MRI of staging 



Proportion of patients 
with rectal cancer and a 
preoperative MRI 
reporting the description 
of the radial margin status 
(mm) 



Number of patients with 
rectal cancer who have 
undergone a preoperative 
MRI reporting the 
description of the radial 
margin status (mm) 



Number of patients 
with colorectal 
cancer undergoing 
initial CT and/or 
MRI examination 



Number of patients 
with low rectal* 
cancer 



Radiology (CT and/or 
MRI examination) report 



Number of patients 
with rectal cancer 
undergoing 
preoperative MRI 



Radiology (MRI 
examination) report, 
discharge letters coming 
from all hospital units/ 
departments (ie, surgery, 
medicine, medical 
oncology, radiation 
oncology) 

Radiology (MRI 
examination) report 



Providing the necessary 
information for a 
comprehensive radiological 
examination, assessment of 
the quality of the flux of 
clinical information 



Planning of further 
diagnostic procedures and 
treatments, 

comprehensiveness of 
diagnostic and staging 
evaluation 



Planning of further 
diagnostic procedures and 
treatments, 

comprehensiveness of 
diagnostic and staging 
evaluation 



62 



Continued 



Table 1 Continued 



Clinical 
domain 



Site Quality indicator 



Numerator 



Denominator 



Medical documentation Rationale 



Reference 



Pathology R 
(n=3) 



Proportion of patients 
with rectal cancer for 
which the request for the 
pathological examination 
includes the information 



Number of patients with 
rectal cancer for which the 
request for the 
pathological examination 
includes the information of 



of neo-adjuvant RT±ChT neo-adjuvant RT±ChT 



Proportion of patients 
with colorectal cancer 
and a sufficient number 
of tumour samples (>3) 



Proportion of patients 
with colorectal cancer 
and a surgical pathology 
report including the 
following characteristics: 
Surgical intervention 
description 
Sample length 
Tumour localisation 
according to WHO 
Tumour size 
Histological type 
according to WHO 
Histological grade 
Resection margins 
Lymph-vascular 
invasion 

Perineural invasion 
Tumour deposits 
(discontinuous 
extramural extension) 
Pathological staging 
(AJCC pTNM) 
Number of retrieved 
lymph nodes 



Number of patients with 
colorectal cancer for 
which 3 or more tumour 
samples were processed 
for the pathological 
analysis 

Number of patients with 
colorectal cancer whose 
pathological report 
includes the following 
characteristics: 

► Surgical intervention 
description 

► Sample length 

► Tumour localisation 
according to WHO 

► Tumour size 

► Histological type 
according to WHO 

► Histological grade 

► Resection margins 

► Lymph-vascular 
invasion 

► Perineural invasion 

► Tumour deposits 
(discontinuous 
extramural extension) 

► Pathological staging 
(AJCC pTNM) 

► Number of retrieved 
lymph nodes 



Number of patients 
with rectal cancer 
undergoing 
neo-adjuvant RT 
±ChT and surgeryt 



Number of patients 
with colorectal 
cancer undergoing 
surgeryt 



Number of patients 
with colorectal 
cancer undergoing 
surgeryt 



Request form of 
pathology examination, 
surgical pathology report 



Surgical pathology report 



Surgical pathology report 



Providing the necessary 
information for a 
comprehensive pathological 
examination, assessment of 
the quality of the flux of 
clinical information 

Comprehensiveness of 
pathology examination 



Proposed by 
the QC 3 
CRC-WG 



Comprehensiveness and 
standardisation of surgical 
pathology report, 
comprehensiveness of 
staging evaluation, planning 
of further treatments 




Continued 



Table 1 Continued 



Clinical 
domain 



Site Quality indicator 



Numerator 



Denominator 



Medical documentation Rationale 



Reference 



► Treatment effect 

► Macroscopic integrity 
of the mesorectum (for 
the rectum only) 
(this quality indicator 
should be provided for 
each characteristic) 

Treatment C&R Proportion of patients 
(n=16) with colorectal cancer 

who have been operated 
in emergency! 

C&R Proportion of patients 
with colorectal cancer 
and dead within 30 days 
and 6 months from the 
surgery (postoperative 
mortality) 



C&R Proportion of patients 
with colorectal cancer 
and postoperative 
multidisciplinary 
discussion 

Proportion of patients 
with malignant rectal 
polyp (pT1) and complete 
endoscopic polypectomy 



Proportion of patients 
with low rectal* cancer 
who have undergone a 
surgical intervention with 
sphincter preservation 



► Treatment effect 

► Macroscopic integrity of 
the mesorectum (for 
the rectum only) 



Number of patients with 
colorectal cancer who 
have been operated in 
emergency! 



Number of patients 
with colorectal 
cancer undergoing 
surgeryt 



Radiology and surgery 
report/discharge letter, 
surgical pathology report 



Assessment of the patient's 
take charge 



65-67 



Number of patients with Number of patients Surgery report/discharge Assessment of the quality 



colorectal cancer and 
dead within 30 days from 
the surgeryt 
Number of patients with 
colorectal cancer and 
dead within 6 months from 
the surgery 

Number of patients with 
colorectal cancer for 
which there has been a 
multidisciplinary 
discussion after surgeryt 

Number of patients with 
malignant rectal polyp 
(pT1) who have 
undergone a complete 
endoscopic polypectomy 

Number of patients with 
low rectal* cancer who 
have undergone a 
surgical intervention with 
sphincter preservation 



with colorectal 
cancer undergoing 
surgeryt 



Number of patients 
with colorectal 
cancer undergoing 
surgeryt 



Number of patients 
with malignant 
rectal polyp (pT1) 



Number of patients 
with low rectal* 
cancer undergoing 
surgeryt 



letter, surgical pathology 
report, access to regional 
Office of Population 
Registry Rosters for the 
assessment of patients' 
vital status 



Surgery, oncology, 
radiation oncology 
reports/discharge letters, 
multidisciplinary 
discussion documentation 

Endoscopy report, 
endoscopic pathology 
reports 



Surgical pathology report, 
surgery report/discharge 
letter 



of surgical procedure 



Planning of further 
diagnostic procedures and 
treatments 



Assessment of the quality 
of surgical procedure 



Assessment of the quality 
of surgical procedure 



72 73 



Proposed by 
the QC 3 
CRC-WG 



7 74-76 



Table 1 Continued 



Clinical 
domain 



Site Quality indicator 



Numerator 



Denominator 



Medical documentation Rationale 



Reference 



R Proportion of patients 
with rectal cancer 
undergoing TEM with RO 
resection 

C&R Proportion of patients 
with colorectal cancer 
and a number of resected 
lymph nodes >12 



C&R Proportion of patients 
with colorectal cancer 
operated on with free 
margins 

C&R Proportion of patients 
with colorectal cancer 
and AJCC TNM clinical 
stage I (from T2N0M0) to 
III (any T, N1M0) 
undergoing a surgical 
resection with 
anastomosis 

Proportion of patients 
with colon cancer and 
AJCC TNM stage II 
(T3N0M0, T4N0M0) 
high-risk (presence of at 
least one of the following 
factors: LN<12, G3, 
lymph-vascular or 
perineural invasion, 
tumour obstruction, 
tumour perforation, pT4) 
or III undergoing adjuvant 
ChT 



Number of patients with 
rectal cancer who have 
undergone TEM with RO 
resection 

Number of patients with 
colorectal cancer with a 
number of resected lymph 
nodes >12 



Number of patients with 
colon cancer who have 
undergone surgery and 
have free margins 

Number of patients with 
colorectal cancer and 
AJCC TNM clinical stage I 
(from T2N0M0) to III (any 
T, N1M0) who have 
undergone a surgical 
resection with 
anastomosis 

Number of patients with 
colon cancer and AJCC 
TNM stage II (T3N0M0, 
T4N0M0) high-risk 
(presence of at least one 
of the following factors: 
LN<12, G3, 
lymph-vascular or 
perineural invasion, 
tumour obstruction, 
tumour perforation, pT4) 
or III, who have 
undergone adjuvant ChT 



Number of patients 
with rectal cancer 
undergoing TEM 



Number of patients 
with colorectal 
cancer undergoing 
surgeryt, but no 
neo-adjuvant 
therapy 

Number of patients 
with colorectal 
cancer undergoing 
surgeryt 

Number of patients 
with colorectal 
cancer and AJCC 
TNM stage I (from 
T2N0M0) to III 



Number of patients 
with colon cancer 
and AJCC TNM 
stage II high-risk or 
III, undergoing 
surgeryt 



Surgical pathology report, 
surgery report/discharge 
letter 



Surgical pathology report, 
surgery report/discharge 
letter 



Surgical pathology report, 
surgery report/discharge 
letter 



Radiology report, surgical 
pathology report.surgery 
report/discharge letter 



Radiology report, surgical 
pathology report, surgery, 
oncology reports/ 
discharge letters 



Assessment of the quality 
of surgical procedure 



Assessment of the quality 
of surgical procedure and 
pathology examination 



Assessment of the quality 
of surgical procedure 



Assessment of the quality 
of surgical procedure 



7 14 16 40 41 80- 
85 



40 41 86 87 



Assessment of the quality 
of oncological treatment 



16 18 40-41 88-91 



Continued 



Table 1 Continued 



Clinical 
domain 



Site Quality indicator 



Numerator 



Denominator 



Medical documentation Rationale 



Reference 



C Proportion of patients 
with colon cancer AJCC 
TNM stage II high-risk or 
stage III undergoing 
adjuvant ChT within 
8 weeks from surgical 
resection 

C&R Proportion of patients 
with colorectal cancer 
and histology of the 
primary tumour or 
metastases obtained 
before the beginning of 
ChT 



Number of patients with 
colon cancer and AJCC 
TNM stage II high-risk or 
III, who have undergone 
adjuvant ChT within 
8 weeks from surgical 
resection 

Number of patients with 
colorectal cancer and 
histology of the primary 
tumour or metastases 
obtained before the 
beginning of ChT 



Number of patients 
with colon cancer 
and AJCC TNM 
stage II high-risk or 
III undergoing 
surgeryt and 
adjuvant ChT 

Number of patients 
with colorectal 
cancer undergoing 
primary ChT 



Radiology report, surgical 
pathology report, surgery, 
oncology reports/ 
discharge letters 



Radiology and pathology 
reports, oncology report/ 
discharge letter 



Assessment of the quality 
of oncological treatment 



Assessment of the quality 
of oncological treatment 



40 41 



C&R Proportion of patients 
with colorectal cancer 
and unresectable 
metastases undergoing 
first-line ChT or bio-ChT 



Number of patients with 
colorectal cancer and 
unresectable metastases 
who have undergone a 
first-line ChT or bio-ChT 



Number of patients 
with colorectal 
cancer and 
unresectable 
metastases 



Radiology and pathology 
reports, oncology report/ 
discharge letter 



Assessment of the quality 
of oncological treatment 



C&R Proportion of patients 
with colorectal cancer 
and hepatic metastases 
primarily unresectable 
turned into resectable 
metastases after 
neo-adjuvant ChT 



Number of patients with 
colorectal cancer and 
hepatic metastases 
primarily unresectable 
turned into resectable 
metastases after 
neo-adjuvant ChT 



Number of patients 
with colorectal 
cancer and 
unresectable 
hepatic metastases 
undergoing 
neo-adjuvant ChT 



Radiology report, 
oncology report/discharge 
letter 



R Proportion of patients Proportion of patients with Number of patients Endoscopic pathology 



with locally advanced locally advanced rectal 

rectal cancer (T3-4 and/or cancer (T3-4 and/or any 

any T, N+ and MO) T, N+ and MO) who have 

undergoing neo-adjuvant undergone neo-adjuvant 

RT±ChT RT±ChT 



with locally 
advanced rectal 
cancer undergoing 
surgeryt 



report, radiology report 
radiation oncology and 
oncology reports/ 
discharge letters 



Proportion of patients 
with rectal cancer and 
undergoing neo-adjuvant 



Number of patients with 
rectal cancer who have 
undergone neo-adjuvant 



Number of patients 
with rectal cancer 
undergoing 



Endoscopic pathology 
report, radiology report, 
radiation oncology and 



Assessment of the quality 
of oncological treatment 



96 




Assessment of the quality 
of oncological and 
radio-oncological treatment 



Assessment of the quality 
of oncological and 
radio-oncological treatment 



Continued 



Table 1 Continued 



Clinical 
domain 




Site Quality indicator 



Numerator 



Denominator 



Medical documentation Rationale 



Reference 



Outcome 
(n=2) 




RT±ChT operated within 
6-8 weeks after the end 
of neo-adjuvant RT±ChT 



C&R Analysis of overall 

survival at 1, 3, 5 and 
10 years from diagnosis 



RT±ChT and were 
operated within 6- 
8 weeks after the end of 
neo-adjuvant RT±ChT 

Number of patients with 
colorectal cancer who 
survive at 1, 3, 5 and 
10 years from diagnosis 



neo-adjuvant RT 
±ChT followed by 
surgeryt 



Number of patients 
with colorectal 
cancer 



C&R Analysis of disease-free 
survival at 1, 3, 5 and 



Number of patients with Number of patients 

colorectal cancer who are with colorectal 

1 0 years from the curative disease-free at 1 , 3, 5 and cancer curatively 

treatment 1 0 years from the curative treated 

treatment 



oncology reports/ 
discharge letters, surgical 
pathology report 



Access to regional Office 
of Population Registry 
Rosters for the 
assessment of patients 
vital status 

Reports/discharge letters 
coming from all hospital 
units/department (ie, 
surgery, medicine, 
oncology, radio-oncology) 




*l_ow rectum: 4-7.5 cm from the dentate line. 100 
tSurgery excludes endoscopic resection and colostomy. 
^Emergency: within 24 h from the onset of symptoms. 

ACR, American College of Radiology; AJCC, American Joint Committee on Cancer; C, colon; C&R, colon-rectum; ChT, chemotherapy; CT, computed tomography; MRI, magnetic resonance 
imaging; R, rectum; RT, radiotherapy; TEM, transanal endoscopic microsurgery; WHO, World Health Organization. 
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and staging, reliable evaluation of the tumour localisation 
and local invasion and particularly for the rectal cancers, of 
a feasible and effective surgery. The first indicator of the 
'diagnosis' group is important to understand what happens 
in a territory where there is no organised screening pro- 
gramme for CRCs, but only an opportunistic screening 
strategy Actually, the tumour is detected because the phys- 
ician submits the patients older than 50 years old to a fecal 
occult blood test (FOBT) or colonoscopy control or if a 
patient, being aware of the possible risk, asks his family 
doctor to undergo screening examinations. These are 
interesting data to be evaluated, also in the hypothesis of a 
CRC screening programme implementation. We therefore 
believe that a higher proportion of patients diagnosed 
through screening (FOBT or colonoscopy in asymptomatic 
patients) would represent a higher diagnostic quality, since 
the therapeutic approach and, consequently, the patients' 
outcome (in terms of recurrence and survival) would be 

OA Q^f OA 

more favourable, as reported in the literature. The 
pathology QIs reflect the importance of good communica- 
tion between clinicians and pathologists in terms of the 
patient's clinical history and consequent evaluation of the 
effectiveness of a neo-adjuvant therapy; moreover, there is 
a need for standardisation of the pathological report fol- 
lowing international guidelines (eg, take at least three 
samples of the tumour during macroscopy), not leaving 
any items unexplained or implicit. In particular, the third 
QI reported in table 1 ( pathology section) refers to the sur- 
gical pathology report, which derives from the surgical 
curative intervention and should be as complete as possible 
to be useful for the future decision about the patient's 
treatment. Our intent is to calculate it not only for all the 
listed items considered together, but also for each item ana- 
lysed individually: for example, the proportion of patients 
with CRC and a definitive pathological report including 
the surgical intervention description; the proportion of 
patients with CRC and a definitive pathological report 
including the tumour size; the proportion of patients with 
CRC and a definitive pathological report including the 
resection margins; the proportion of patients with CRC 
and a definitive pathological report including the patho- 
logical staging (AJCC pTNM); etc. The treatment QIs 
cover the general issues of surgery, such as emergency, post- 
operative mortality and a multidisciplinary discussion of 
the clinical case; furthermore, they focus on the debate of 
the retrieved lymph nodes, on the timing between radio- 
therapy and surgery, on the adjuvant chemotherapy and 
on the attitude towards the metastatic patients. The two 
main items of the outcome chapter refer to the overall and 
disease-free survival. Although it is necessary to wait for a 
certain follow-up period (ie, 1, 3, 5-10 years from the date 
of diagnosis for the calculation of overall survival, and from 
the date of curative treatment for the calculation of disease- 
free survival), they will represent the overall resume of the 
diagnostic and treatment quality of CRC patients. Our 
intent will be to analyse the overall and disease-free survival 
according to some of the proposed QIs (such as QI con- 
cerning the pathological characteristics of the tumours, QI 



of the adjuvant chemotherapy in patients with colon 
cancer and AJCC TNM stage II high-risk or III, QI of colo- 
rectal patients operated on with free margins, QI of locally 
advanced rectal cancer patients undergoing neo-adjuvant 
radio±chemo therapy, etc). We will finally compare our 
results with other regional and national reality, favouring 
the interpretation of each single QI. Concerning the QIs 
about follow-up, AB did not finally include any of them. 
Indeed, although the follow-up procedures are suggested 
by several international guidelines, they are based on levels 
II— III evidence and controversies remain regarding the 
selection of optimal strategies for following up patients 
after potentially curative CRC surgery. 40-43 

The first limitation of the current work is the level of 
evidence found in the literature. For some QIs, strong 
evidence of their validity was not available from rando- 
mised controlled trials. However, this situation is 
common to many aspects of healthcare, and it was the 
very reason that the expert panel methodology was 
developed — specifically, to identify the processes that are 
most likely to be valid measures of quality when the 
highest level of evidence is not available. 19 23 44 Second, 
we may have missed some studies during the literature 
search, and consequently, some QIs may not have been 
proposed to the QC 3 CRC-WG since the beginning of 
the QIs revision process. However, this limitation should 
have been overcome by the fact that the members of the 
QC 3 CRC-WG were likely to be very familiar with the lit- 
erature, and had the opportunity to suggest other QIs 
based on their experience and literature search. 7 27 28 45 
Thus, we integrated the best research evidence with clin- 
ical expertise, as reported by Sackett et A further 
limit could be the feasibility of measuring QIs in terms 
of data collection and calculation, which is immediately 
the next step. Actually, the QIs selected by both the QC 3 
CRC-WG and the AB represent an ideal set of criteria to 
measure the quality of CRC care; at the same time, they 
both were concerned about the feasibility, validity and 
reliability of clinical data collection, necessary for the 
calculation of each single QI at the population-based 
level. This is the reason why most of the identified QC 3 
QIs are common to many QoCC studies. 1 247 14-20 
Besides the traditional Delphi process, the panellists had 
the chance to comment on the single QI from a 
population-based cancer registration and data collection 
point of view. Therefore, in order to warrant an accurate 
measurement, those QIs reaching more than 70% of the 
agreement, confirming their scientific and clinical value, 
but evaluated at least by one of the experts as not feas- 
ible and difficult to be collected at the population-based 
level, were definitely excluded. In addition, we per- 
formed a retrospective preliminary pilot collection on 
the detailed and necessary incidence data of CRC which 
occurred in 2011, realising that the measurement of 
most QIs is feasible, whereas for some selected QIs the 
retrieval of variables would need additional efforts; some 
preliminary results were presented in national and inter- 
national conferences and congresses, receiving positive 
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feedback in both the clinical and epidemiological set- 
tings. 47-50 Only the definitive results will give us the pro- 
portion of missing information, whose magnitude will be 
assessed. 

The selected QC 3 CRC QIs will be applied in a 
population-based setting, where age is an extremely 
important determinant of treatment. The elderly are rarely 
included in the randomised clinical trials with the conse- 
quence of a possible 'underuse of treatment'. 25 51 52 At a 
broad European level, national audit registries in surgical 
oncology have led to improvements with a great impact 
and they offered the possibility, as for our project, to 
perform research on patients that are usually excluded 
from clinical trials such as the elderly and comorbid 
patients. 53 54 Evidence suggests that the relative benefits of 
treatment for the elderly are similar to those seen for 
cancer patients in general, though decision-making for 
treatment becomes more complex as life expectancy, coex- 
isting illnesses and functional status all need to be consid- 
ered. 25 51 52 Applying these QIs, and if all these items will 
be satisfied, we can affirm having a real good quality 
process of CRC care for the whole population. The foresee- 
able future in quality evaluation and improvement of 
healthcare will quite likely involve a more frequent use of 
QIs by regulatory and accrediting agencies, stakeholders, 
clinicians, individual hospitals and healthcare providers, as 
well as patients. This underlines the fact that QIs should be 
defined, developed and tested with scientific evidence- 
based rigour in a careful and transparent manner, taking 
into account their degree of relevancy, validity, reliability 
and feasibility. 30 32 Although QIs have been defined in 
several different ways, all authors agreed that the final aim 
was the improvement of patients outcome. 

The systematic trend analysis of QIs allows the assess- 
ment of immediate changes and improvements in the 
diagnostic-therapeutic process, which could be trans- 
lated into a short-term benefit for patients, without 
waiting for survival analysis typically needing some years 
to produce, because of the patients' follow-up. 
Furthermore, this system of evaluation and autoevalua- 
tion could favour the surveillance and monitoring of the 
comprehensive level of oncological care in the region, 
the clinical performance homogeneity, the possible 
weakness of the clinical network and finally the correct- 
ive interventions to be adopted to improve QoCC. 

With this study, we hope to increase the awareness of the 
value of QIs in healthcare so as to encourage more 
uniform practices and improve provider documentation of 
medical care in our region; moreover, we hope that the 
standardisation of QIs among different regions will help to 
define the threshold of a minimal standard of care. 

Correction notice This paper has been corrected since it was first published. 
One of the authors, Andrea Bordoni, did not have an affiliation number after 
their name. 
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